Repository navigation
Use CodeMirror in ruleset debugger (#11794) - #13560
Conversation
|
It doesn't look like we use npm in the build process, for example we don't require it in |
|
@jeremyn You used |
|
I cannot reproduce the |
|
I can reproduce a CSS file but I had to delete an empty line at the end. |
|
I used https://www.npmjs.com/package/uglifyjs as mentioned in the git commt message. |
|
I'm not sure what the relationship is between https://www.npmjs.com/package/uglifyjs and https://www.npmjs.com/package/uglify-js -- I see they share a collaborator -- but https://www.npmjs.com/package/uglify-js has many more downloads so I agree that's the one to use. I've (force-)pushed an update that uses |
|
@koops76 I'm not clear from your comments whether you could reproduce the CSS minification. Try the following: Do you get the same hashes? |
|
@jeremyn Use |
|
I reproduced everything. |
|
I do think if we're adding dependencies (which I'm not necessarily against, as the We already have @koops76 I see you deleted comments that suggested similar. I'm curious why? |
|
@Hainish I agree that listing a dependency in a However assuming we prefer the By the way, @koops76 recently opened an issue (#13512) to discuss remaking the build system. |
|
@jeremyn my typical workflow would be to create a |
|
I think it's too early to create a project-wide installation architecture, just to make CodeMirror available to the ruleset debugger. It raises questions like, where should the package file go? How should we document the installation process? Also for security we should include checksums somewhere, and I don't think Maybe you're concerned that the only documentation about how to create the minified CodeMirror files is in the git commit message, which isn't great. I can add a small |
|
@jeremyn NPM is better than Yarn security-wise. It creates a |
👍 |
|
@koops76 According to https://docs.npmjs.com/files/package-lock.json#integrity : "For registry sources, this is the "dependencies": {
"uglifycss": {
"version": "0.0.27",
"resolved": "https://registry.npmjs.org/uglifycss/-/uglifycss-0.0.27.tgz",
"integrity": "sha1-U1U7gRXeJtzle0K6CzTojdgNDN4="
}
}If you change the This is not all to say "you're wrong", but instead just that npm integrity checking looks complicated. I'm also concerned that it silently continues to work even with a bad hash. I don't know how Yarn would compare in similar tests. |
|
And also, it would be nice if we could just merge this PR without having to first dig deep into comparing the integrity mechanisms of npm vs Yarn vs anything else. |
|
@jeremyn This is because uglifycss is too old and doesn't have a SHA512 hash yet. |
|
@jeremyn I think it's sensible to merge in these dependencies without creating a dependency management system in general. I haven't had a chance to review this yet (I'm in meetings all day) but after just a cursory look, can I suggest we put any external deps in a separate path, say |
I mean this is sensible for now, given the relatively confined scope of the usage of this library. I do think it will be necessary to have a dependency management system in the future, as we build out the extension. I also think if we're not to go full-blown dependency system right now, having this under a |
|
@Hainish I think the directory should be called |
EFF staff should review the CodeMirror license (MIT) at https://raw.githubusercontent.com/codemirror/CodeMirror/5.31.0/LICENSE and make any necessary updates to comply with the license.
|
I moved the minified files from I used a force push instead of a regular push. I'm sorry for the trouble. This should keep unnecessary versions of the minified files out of the repository. I prefer @Hainish For licensing, as I describe in issue #13062 (comment) (still waiting for a response there), I have concerns about the overall licensing of this project. That being the case, you/the EFF can handle the CodeMirror licensing. In fact I don't want your licensing changes to be done under my name, so I've unchecked the "Allow edits from maintainers" option to make sure you don't push licensing changes and then accidentally squash-and-merge them. (I know you would not do that intentionally if I asked you not to, but you might forget if this issue drags on.) So that will need to be done in a separate PR. |
|
@jeremyn thanks for those changes. As I mentioned I'm in meetings all day so I'll have to wait until at least tomorrow to review, but I definitely plan to have this merged by next release (which I'm thinking should be early next week.) |
Hainish
left a comment
There was a problem hiding this comment.
Looking good. I've reproduced the codemirror js and css and get the same results. A few final changes and then I'm ready to merge.
| ## CodeMirror 5.31.0 | ||
|
|
||
| ``` | ||
| $ npm install uglify-js@3.1.9 |
There was a problem hiding this comment.
Since this is not a package dependency and is intended to be run as a command line app, let's add a -g flag for global install as the installation instructions specify: https://github.com/mishoo/UglifyJS2#install
There was a problem hiding this comment.
What's the drawback of defining all the dependencies in package.json in desired versions and define npm scripts that would execute proper commands?
Global packages should be avoided when possible. Someone may skip those steps as they've already performed them before and not notice they've since installed different versions since some other project required different ones. Using local packages gets rid of that problem.
There was a problem hiding this comment.
Good point, but the above command will cause uglify-js to be added to the dependencies hash, rather than the devDependencies hash. Thus we should preference running this with the -D flag, and the below reproducibility instructions should be modified to be:
../node_modules/uglify-js/bin/uglifyjs
There was a problem hiding this comment.
Is there any reason to specify those packages directly instead of just having package.json checked into the repo and running npm install?
BTW, if you're referring to the binary by path, it's usually better to use ../node_modules/.bin/uglifyjs as that depends less on the internal package structure. node_modules/.bin contains links to binaries exported by top-level packages.
There was a problem hiding this comment.
@mgol creating a dependency management system is on the table, but I don't want to block merging until we have one in place. See #13560 (comment) and resulting discussion.
There was a problem hiding this comment.
These instructions are supposed to give someone who is knowledgeable about npm the key information needed to reproduce the files: package names, version numbers, options used. It's not supposed to completely walk a beginner through the process of making these files. The user can decide for themselves whether to install the minifier executables globally or not and how to get the files into whatever directories they want.
|
|
||
| ``` | ||
| $ npm install uglify-js@3.1.9 | ||
| $ npm install uglifycss@0.0.27 |
There was a problem hiding this comment.
|
|
||
| ```bash | ||
| uglifycss \ | ||
| --output codemirror-5.31.0.min.css |
| @@ -0,0 +1 @@ | |||
| external/* | |||
There was a problem hiding this comment.
We'll want to add chromium/.eslintignore to the top-level .build_exclusions as well.
| <link href="debugging-rulesets.css" rel="stylesheet"> | ||
| <script src="external/codemirror-5.31.0.min.js"></script> | ||
| <link href="external/codemirror-5.31.0.min.css" rel="stylesheet"> | ||
| <script src="external/codemirror-5.31.0.xml.min.js"></script> |
There was a problem hiding this comment.
I may have not been specific enough in my suggestion. Can we move each of these under external/codemirror/, just so each dependency moving forward is in its own neat path?
| --compress \ | ||
| --mangle \ | ||
| --comments '/copyright|license/' \ | ||
| --output codemirror-5.31.0.min.js \ |
There was a problem hiding this comment.
Relating to the suggestion above of adding each external dependency to its own unique path, we'll want to change these output paths to external/codemirror/...
|
Thinking it over, I don't really see the use for minification here. In web development, it's useful to minify in order to reduce the size of the downloaded resources. Here, we're already compressing the extension at build time, since If there isn't a definitive benefit to minification here, we could remove the necessity for |
|
@Hainish It's a common courtesy to minify any distributed files to save some bits, I think the extension's code should also be minified. |
|
I do see some savings in download size between the minified
and non-minified:
so it appears that minification does save us some space. |
|
@jeremyn We will add a minification step later, you may completely remove minification for now. |
|
@koops76 let's leave minification in if it's a win on download size. |
|
@Hainish I mean minification for the whole extension. |
|
I kept the minified files. Someone who wants to see the originals can refer to the new |
@Hainish @cowlicks This should resolve issue #11794 to add CodeMirror to the ruleset debugger.
The commit message describes how I uglified (minified) the CodeMirror files. For security purposes I encourage someone at the EFF to verify these minifications.
Also, as the commit message says, this almost certainly requires another update to satisfy CodeMirror's MIT license. In #13062 (comment) I discuss some concerns I have about HTTPS Everywhere's approach to licensing, so for the time being I will let someone at the EFF figure out how to handle licensing here. Please feel free to push any needed changes directly to my branch.